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Summary 


The  research  project  was  aimed  at  three  broad  approaches  to  development  and 
evaluation  of  new  methods  to  assess  perceptual  speed  abilities:  The  first  approach  expands 
the  previous  research  on  perceptual  speed  abilities  to  consider  the  benefits  of  practice  and  skill 
acquisition  for  refined  assessment  of  asymptotic  perceptual  speed  abilities  and  evaluate  the 
role  of  interindividual  and  intraindividual  differences  in  speed-accuracy  tradeoff  within 
assessment  of  perceptual  speed  abilities.  The  second  approach  takes  advantage  of 
computerized  devices  for  assessment  of  a  series  of  perceptual  speed  abilities,  especially  with 
the  investigation  of  alternative  display  and  input  devices  (e.g.,  touch-sensitive  computer 
monitors).  The  third  approach  evaluates  the  new  test  procedures  with  respect  to  two 
important  validation  criteria:  (a)  comparison  with  extant  measures  of  abilities  (including 
measures  of  working  memory,  general  ability,  and  psychomotor  abilities);  and  (b)  an 
evaluation  of  the  efficacy  of  perceptual  speed  ability  measures  for  predicting  individual 
differences  in  task  performance.  The  goal  of  this  work  has  been  to  demonstrate  the  feasibility 
of  using  new  techniques  and  technology  for  assessment  of  perceptual  speed  abilities,  to 
provide  an  integrated  approach  with  psychomotor  and  cognitive  abilities,  and  to  demonstrate 
the  validity  of  new  measures  of  perceptual  speed  abilities  for  prediction  of  criterion  task 
performance.  In  this  context,  we  developed,  refined,  and  demonstrated  both  the  reliability 
and  validity  of  a  set  of  perceptual  speed  ability  tests.  These  tests  are  fully  computerized  and 
can  be  administered  in  an  offithe-shelf  PC  computer  environment.  The  individual  tests  are 
short  (each  test  requires  only  a  few  minutes  to  administer),  and  are  robust  to  differential 
strategies  of  speed  or  accuracy  emphasis.  With  the  existing  software  platform,  the  tests  can 
be  modified  with  little  effort  (which  with  further  development,  could  be  modified  in  real¬ 
time),  to  avoid  any  problems  associated  with  repeated  assessments  or  test  practice.  The 
measures  demonstrate  substantial  incremental  validity,  over  and  above  the  kind  of  extant  tests 
of  cognitive  and  intellectual  abilities  that  underlie  the  standard  selection  measures  used  by  the 
U.S.  Air  Force.  Further  exploration  of  these  tests,  or  tests  similar  to  those  developed  in  this 
project,  is  highly  recommended  for  ultimate  adoption  in  the  operational  selection  environment. 
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I.  Introduction 

This  project  involves  the  continuation  of  a  line  of  theoretical  and  empirical  research 
that  will  always  be  of  great  importance  to  the  optimal  functioning  of  the  U.S.  Air  Force, 
namely  the  measurement  of  abilities  that  are  of  critical  importance  for  skill  learning,  both  at 
initial  phases  of  training,  and  later  on-the-job  performance.  This  has  been  a  central  issue  for 
selection,  training,  and  classification  purposes,  even  before  formal  the  existence  of  a  separate 
U.S.  Air  Force  (e.g.,  the  extensive  work  condueted  by  the  U.S.  Army  Air  Forces  Aviation 
Psychology  Research  Program,  under  the  direction  of  John  Flanagan  (see  Flanagan,  1948; 
Guilford  &  Lacey,  1947;  Melton,  1947).  Over  the  past  60  or  so  years,  efforts  have  been 
devoted  to  developing  and  validating  taxonomic  representations  of  cognitive  and  intellectual 
abilities.  However,  more  research  is  still  needed,  given  the  complexity  of  human  cognitive 
processes,  and  the  limited  resourees  that  have  been  available  for  their  investigation.  The 
current  effort  has  focused  on  one  major  area  of  human  capabilities  --  perceptual  speed  abilities 
—  that  have  received  relatively  little  researeh  attention,  but  have  been  shown  to  account  for 
substantial  variance  in  the  performance  of  complex  skill  tasks,  where  speed  and  accuracy  of 
responding  are  critical  for  success. 

From  the  1970s  through  the  1990s,  the  dominant  theme  in  the  assessment  of  individual 
differences  in  abilities  research  has  been  an  adaptation  of  the  “information-processing 
approach.”  This  approach  to  abilities  research  started  with  the  work  by  Hunt  and  his 
colleagues  (Hunt,  Frost,  &  Lurmeborg,  1973),  followed  by  Sternberg  (1977),  Carroll  (1980), 
and  culminated  in  the  large-seale  studies  xmder  the  Learning  Abilities  Measurement  Program 
(LAMP)  at  the  Air  Force  Research  Laboratory  (e.g.,  see  Kyllonen,  1985,  1991;  Kyllonen  & 
Christal,  1989;  --  see  also  Carroll,  1993  for  an  extensive  review).  While  these  research  efforts 
have  added  substantially  to  the  corpus  of  literature  on  the  taxonomic  nature  of  human 
abilities,  it  may  be  argued  that  in  the  aggregate,  these  efforts  have  not  substantially  enlarged 
the  extant  measures  for  selection  and  classification  of  entry-level  recruits  (namely,  the 
ASVAB),  which  are  dominated  by  an  assessment  of  general  ability  (g)  (e.g.,  see  Olea  &  Ree, 
1994;  Ree  &  Earles,  1991).  Little  additional  predictive  validity  is  likely  to  be  obtained  by 
developing  improved  assessments  of  g,  or  working  memory  (for  discussions  of  working 
memory  and  other  abilities,  see  Baddeley,  1986;  Cantor,  Engle,  &  Hamilton,  1991;  Engle,  et 
al.,  1999;  Kyllonen  &  Christal,  1990).  Similarly,  little  gain  in  predictive  validity  is  expected 
from  inclusion  of  measures  of  basic  information  processing  (Ackerman,  2000;  Carroll,  1980, 
1993).  However,  there  are  two  additional  domains  of  human  ability  assessment  that  have 
shown  promise  for  improving  prediction  of  individual  differences  in  performance,  discussed  in 
turn  below. 

The  first  domain  concerns  assessment  of  psychomotor  abilities  (e.g.,  Ackerman  & 
Cianciolo,  1999,  2000;  Kyllonen,  1994).  This  research  has  suggested  that  psychomotor 
abilities  measures  may  yield  substantial  incremental  validity  for  predicting  individual 
differences  in  skill  acquisition  and  task  performance.  From  a  pragmatic  perspective,  there 
remain  issues  of  the  feasibility  of  incorporating  psychomotor  ability  measures  into  large-scale 
testing  programs  —  mainly  because  of  the  need  for  computers  for  administration  purposes. 
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While  the  initial  research  has  been  promising,  additional  research  is  needed  to  hone  the 
measures  of  psychomotor  abilities,  with  a  view  toward  implementation. 

The  second  domain  which  appeared  highly  promising,  is  the  area  of  perceptual  speed 
ability  assessment.  In  previous  AFOSR-sponsored  research  (e.g.,  Ackerman,  1999), 
taxonomic  theoretical  developments  and  empirical  research  have  converged  on  a  multiple 
factor  representation  of  perceptual  speed  abilities.  This  research  has  further  demonstrated  that 
there  is  indeed  much  promise  in  using  selected  perceptual  speed  ability  measures  in  predicting 
individual  differences  in  skill  acquisition  and  task  performance.  The  current  research  project 
concerned  the  further  exploration  of  the  perceptual  speed  ability  domain,  and  has  as  a  major 
goal  the  demonstration  of  the  feasibility  of  perceptual  speed  ability  assessments  within  both 
paper  and  pencil  and  computerized  frameworks. 

Specific  Backgroimd 

The  term  “perceptual  speed  ability”  is  somewhat  of  a  misnomer,  from  the  perspective 
of  experimental  psychology,  because  perceptual  speed  tests  involve  all  phases  of  information 
processing,  from  encoding,  perception,  central  processing/working  memory,  response  selection 
and  execution.  At  the  most  general  level,  what  characterizes  perceptual  speed  tests 
(Ackerman  &  Cianciolo,  2000)  is  that  test  items  are  relatively  simple  -  examinees  are  able  to 
correctly  respond  to  each  item  without  error  in  untimed  testing.  Rather,  the  key  to 
performance  on  each  such  test  is  the  speed  and  accuracy  of  responding.  The  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB)  contains  two  tests  that  at  least  partly  tap  aspects  of 
perceptual  speed,  namely  Code  Speed  and  Numerical  Operations.  These  tests,  though,  appear 
to  barely  scratch  the  surface  of  the  imderlying  construct.  From  our  taxonomic  research,  we 
suspect  that  these  are  not  optimal  tests  for  a  variety  of  application  purposes. 

While  there  have  been  many  so-called  perceptual  speed  ability  assessments  developed 
over  the  past  80  years  or  so,  the  nature  of  such  factors  has  been  largely  imexplored.  Carroll 
(1993,  p352),  for  example,  in  his  review  of  the  corpus  of  human  cognitive  abilities  only 
reports  a  few  limited  studies,  and  suggested  that  “as  far  as  can  be  determined  from  the  data 
[test  design  differences]  do  not  make  for  systematic  differences  in  factor  loadings.”  Until  our 
recent  AFOSR-sponsored  research,  there  has  only  been  one  large-scale  investigation  into 
perceptual  speed  abilities  —  namely,  the  work  of  Guilford  and  his  colleagues  in  the  U.S. 

Army  Air  Forces  Aviation  Psychology  Research  Program  (Guilford  &  Lacey,  1947).  While 
that  research  program  was  highly  productive  in  developing  selection  tests  for  pilots,  navigators 
and  gunners,  very  little  taxonomic  work  was  reported  from  those  investigations.  Rather,  when 
a  test  was  developed  that  showed  an  improvement  in  predicting  training  success,  it  was 
rapidly  incorporated  into  the  selection  and  classification  program.  For  example,  the  Dial  and 
Table  Reading  test  was  found  to  be  the  most  effective  test  for  selection  of  pilots  (Guilford  & 
Lacey,  1947,  p.  403).  Subsequent  research,  by  the  Federal  Aviation  Administration  Civil 
Aeromedical  Institute  (Cobb  &  Matthews,  1972;  Sells,  et  al.,  1984)  found  the  Dial  and  Table 
Reading  Test  and  a  similar  test  to  be  highly  effective  in  predicting  training  success  of  civilian 
air  traffic  controllers.  For  somewhat  obscure  reasons,  the  U.S.  Air  Force  dropped  the  use  of 
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this  test  shortly  after  World  War  II,  and  the  FAA  diseontinued  researeh  on  the  test  after  a 
single  study. 

Our  research  (e.g.,  Ackerman  &  Kanfer,  1993a;  Ackerman,  Kanfer,  &  Goff,  1995)  has 
documented  that  the  Dial  Reading  Test  is  both  highly  predictive  of  individual  differences  in 
performance  during  complex  skill  acquisition,  both  in  the  laboratory  and  in  job  selection  — 
largely  supportive  of  the  research  from  the  AAF  and  FAA  programs.  (It  may  be  useful  to 
note  that  Ackerman  &  Kanfer  [1993b]  developed  a  parallel  version  of  the  Dial  Reading  Test, 
and  it  remains  one  of  the  most  effective  tests  in  the  battery  we  developed  for  the  Minnesota 
Air  Traffic  Control  Training  Center  [a  regional  facility  sponsored  by  the  FAA].)  Our  more 
recent  research  has  focused  on  the  taxonomic  underpinnings  of  the  test,  along  with  a  series  of 
other  perceptual  speed  tests,  with  an  aim  to  developing  a  framework  for  the  exploration  and 
specification  of  the  sphere  of  perceptual  speed  abilities.  From  this  work,  the  Dial  Reading 
Test  has  been  shown  to  be  an  excellent  marker  for  Perceptual  Speed  (PS)-Complex  ability, 
one  of  four  factors  underlying  the  larger  domain  of  perceptual  speed. 

Structure  of  Perceptual  Speed  Abilities 

In  a  previously  completed  AFOSR-sponsored  research  project  (Ackerman  &  Cianciolo, 
2001,  2002),  we  created  an  augmented  battery  of  perceptual  speed  ability  measures  that  was 
predicated  on  a  taxonomic  approach  to  the  ability  construct.  Fifteen  new  tests  were  developed 
that  represented  different  contents  (verbal,  spatial,  numerical),  different  levels  of  consistency 
of  stimuli,  and  different  levels  of  stimulus  familiarity.  In  addition,  six  extant  measures  of 
perceptual  speed  ability  were  administered  to  provide  for  calibration  with  the  wider  research 
literature,  along  with  measures  of  general  and  broad  content  cognitive  abilities.  At  one  level, 
this  research  demonstrated  that  there  is  significant  common  variance  among  perceptual  speed 
ability  measures.  At  a  more  fine-grained  analysis,  the  results  indicated  that  there  are  at  least 
three  identifiable  perceptual  speed  factors,  which  we  named  “PS-Pattem  Recognition,”  “PS- 
Scanning,”  and  “PS-Memory.”  As  discussed  above  (see  also  Ackerman  et  al.,  1995),  we 
determined  that  there  also  existed  a  ‘complex  process’  perceptual  speed  ability,  which  we 
identified  as  PS-Complex. 

Ability  Determinants  of  Individual  Differences  in  Skill  Acquisition 

In  previous  research,  we  have  made  substantial  progress  in  first  proposing,  then 
empirically  validating  a  theoretical  approach  to  the  ability  determinants  of  individual 
differences  in  performance  during  skill  acquisition  (e.g.,  Ackerman,  1988,  1990,  1992,  2000; 
Ackerman  et  al.,  1995).  In  the  most  recent  investigations  (Ackerman  &  Cianciolo,  2000, 
2002)  we  have  expanded  the  foundation  of  ability  measures  to  include  a  selection  of 
psychomotor  ability  measures  (e.g..  Serial  RT,  Maze  Tracing,  Mirror  Tracing,  Pursuit)  using  a 
specially  designed  software  suite  for  test  administration  and  PCs  with  touch-sensitive 
computer  monitors.  Initial  validation  efforts  with  a  full  set  of  ability  measures  (verbal, 
spatial,  numerical,  perceptual  speed,  and  psychomotor)  have  supported,  in  particular,  the 
incremental  predictive  validity  of  perceptual  speed  and  psychomotor  measures,  especially  in 
predicting  individual  differences  in  performance  at  intermediate  and  highly  practiced  levels  of 
performance. 
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Other  research  in  this  field  has  demonstrated  that  prediction  of  individual  differences 
in  performance  during  skill  acquisition  and  at  asymptotic  levels  of  performance  can  be 
significantly  and  substantially  improved  (e.g.,  Ackerman  &  Kanfer,  1993a,  1993b;  Ackerman 
&  Cianciolo,  1999,  2001).  By  building  on  the  foundation  of  general  and  broad  content 
cognitive  abilities  (e.g.,  verbal,  spatial,  numerical),  properly  selected  measures  of  perceptual 
speed  and  psychomotor  ability  measures  may  be  used  to  enhance  selection  procedures  and  to 
identify  individuals  who  are  more  likely  to  succeed  or  fail  during  training  or  skill  acquisition 
practice.  While  there  an  extensive  corpus  of  empirical  research  on  the  general  and  broad 
content  abilities,  additional  exploratory  and  confirmatory  research  is  needed  in  the  perceptual 
speed  ability  domain.  Moreover,  to  integrate  perceptual  speed  abilities  into  the  larger 
consideration  of  abilities,  additional  research  is  needed  to  establish  how  particular  aspects  of 
perceptual  speed  abilities  relate  to  cognitive  and  psychomotor  abilities,  and  to  establish  how 
these  abilities  relate  to  skill-learning  task  performance  criteria.  The  experiments  described 
below  represented  an  in-depth  pursuit  of  these  construct  and  criterion-related  validation  goals 
for  perceptual  speed  abilities. 

II.  Current  Research 

Critical  Abilities  for  Performance 

In  the  current  research  project,  and  based  on  our  previous  AFOSR-sponsored  research 
(Kanfer  &  Ackerman,  1990;  Ackerman  &  Kanfer,  1996;  Ackerman,  1999),  we  have 
developed  theory  and  conducted  extensive  empirical  research  with  an  aim  toward  developing  a 
useful  classification  of  abilities  that  are  critical  for  the  prediction  of  individual  differences  in 
performance  during  and  after  skill  acquisition.  In  concert  with  other  U.S.  Air  Force  research 
(e.g.,  Kyllonen,  1985,  1994;  Woltz,  1988)  and  the  extant  literature  on  human  abilities  (e.g., 
see  Carroll,  1993),  we  have  identified  several  key  ability  classes  that  have  been  demonstrated 
to  have  substantial  validity  for  predicting  individual  differences  in  performance  on  skill- 
learning  tasks.  Each  of  these  abilities  is  briefly  discussed  below. 

General/Content  Abilities.  General  intellectual  abilities  have  perhaps  the  most 
substantial  basis  for  predicting  individual  differences  in  learning  tasks  —  starting  with  the 
early  research  of  Binet  &  Simon  (1905)  regarding  the  prediction  of  academic  performance. 
Initial  research  and  theory  (e.g.,  Spearman,  1904)  suggested  that  general  intelligence  was 
amorphous  and  indivisible.  However,  by  the  1920s  and  1930s,  it  had  been  clearly  established 
that  general  intelligence  was  not  all-encompassing  in  terms  of  either  basic  underlying 
individual  differences  (e.g.,  see  Kelly,  1928;  Thomson,  1939;  Thurstone,  1938),  or  in  terms  of 
building  optimal  models  for  predieting  performance  in  specific  domains.  So-called  “group 
factors”  or  “content  abilities”  (e.g.,  spatial,  math,  verbal)  were  reliably  found  when  large 
batteries  of  ability  tests  were  subjected  to  factor  analysis.  Moreover,  examination  of  these 
content  factors  revealed  that  validity  in  predicting  individual  differences  in  performance 
depended  in  part  on  a  match  between  the  content  of  the  predictor  ability  and  the  content  of 
the  criterion  task.  That  is,  when  trainees  were  required  to  learn  highly-spatially  demanding 
tasks  or  jobs,  tests  of  spatial  ability  were  generally  more  highly  eorrelated  with  eriterion  task 
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performance  than  were,  say,  verbal  ability  measures  (e.g.,  see  Guilford  &  Lacey,  1947). 

It  is  possible  to  identify  dozens  of  group  factors,  but  as  shown  by  Snow,  Kyllonen, 
and  Marshalek  (1984),  dividing  the  ability  sphere  into  Spatial,  Math,  and  Verbal  abilities  is  a 
valid  simplification  for  categorization  purposes.  In  addition,  this  breakdown  of  the  structure 
of  abilities  provides  a  simplifying  scheme  that  can  be  adapted  to  particular  broad-based 
prediction  paradigms  (when  the  researcher  wishes  to  generalize  beyond  a  narrow  range  of 
criterion  tasks).  We  refer  to  these  categories  as  content  factors,  because  at  least  on  a  surface- 
level  analysis  of  task  requirements,  the  content  of  the  tests  describes  the  stimuli  upon  which 
the  individual  must  work  with  to  solve  the  test  problems  (i.e.,  words  for  Verbal  content, 
figures  for  Spatial  content,  and  numbers  for  Math  content) 

A  final  general  ability  has  also  been  subjected  to  a  great  amount  of  discussion  in  the 
past  10  years  or  so.  In  one  framework  Working  Memory  (a  construct  from  the  experimental 
psychology  literature)  can  be  thought  of  as  attentional  capacity  (e.g.,  see  Baddeley,  1986). 
Recent  discussion  has  also  portrayed  working  memory  as  essentially  the  same  thing  as 
reasoning  ability  —  which  is  central  to  the  conceptualization  of  general  intelligence  (over  and 
above  respeetive  content  abilities),  see  Kyllonen  &  Christal  (1989,  1990).  Working  Memory 
has  been  implicated  as  critical  for  predicting  individual  differences  in  task  performance, 
especially  on  complex  tasks.  As  such,  a  consideration  of  Working  Memory,  as  a  supplement 
to  the  content  factors  is  be  incorporated  into  two  of  the  empirical  studies  described  below. 

Perceptual  Sneed  Abilities.  Previous  research  in  our  laboratory  has  demonstrated  that 
perceptual  speed  abilities  may  provide  a  critically  important  supplement  to  measures  of 
general  and  broad  content  abilities,  in  the  prediction  of  individual  differences  in  performance 
during  skill  acquisition  (e.g.,  Ackerman,  1988;  1990;  1992;  Ackerman  &  Kanfer,  1993; 
Ackerman,  Kanfer  &  Goff,  1995).  According  to  Ackerman  (1988),  Perceptual  Speed  (PS)  is 
defined  as:  “In  the  language  of  skill  acquisition,  individual  differences  found  on  [perceptual 
speed]  tests  are  directly  attributable  to  the  speed  with  which  these  productions  can  be 
implemented  and  compiled  (e.g.,  see  Werdelin  &  Stjemberg,  1969).  (p.  190)”  It  turns  out 
that  a  more  extensive  analysis  of  extant  measures  of  PS  ability  suggested  that  the  tests  could 
be  taxonomized  on  a  number  of  (overlapping)  dimensions,  based  on  an  information-processing 
analysis  of  test  content.  A  previous  analysis  (Ackerman  &  Rolfhus,  1996)  suggested  the 
following  candidate  dimensions  of  underlying  test  demands:  (a)  Content  (spatial,  verbal, 
numerical),  (b)  Consistency  (consistent  information  processing  or  varied  information 
processing);  (c)  Novelty  (familiar  vs.  novel  stimuli);  (d)  Precision  (of  encoding  and  of 
responding);  (e)  Modality  (of  encoding  and  of  responding);  (f)  Memory  demands  (low  vs. 
high),  (g)  Scanning  vs.  single  items. 

While  it  would  be  theoretically  possible  to  ereate  a  completely-crossed  sampling  of 
tests  for  these  seven  dimensions,  the  exponential  munber  of  tests  that  would  be  generated 
make  for  an  impractical  empirical  evaluation  (especially  inasmuch  as  tests  could  also  combine 
dimensions  —  such  as  a  test  that  had  mixed  spatial  and  verbal  content).  Instead,  in  Ackerman 
&  Cianciolo  (2000)  we  attempted  to  sample  Wadly  from  most  of  these  dimensions,  in  the 
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hope  that  patterns  of  consistent  individual  differences  would  emerge,  inductively,  across  the 
various  dimensions.  In  all,  we  adapted  or  created  15  tests  of  PS  abilities,  to  administer  along 
with  6  extant  measures  of  PS  abilities,  12  tests  of  content  abilities  (verbal,  spatial,  numerical), 
and  a  set  of  Choice  and  Simple  RT  measures. 

By  subjecting  the  PS  tests  to  a  factor  analysis,  we  established  that  there  is  a  general  PS 
factor,  but  there  were  also  three  lower-order  PS  factors  that  were  clearly  differentiable  in 
terms  of  the  information  processing  demands  of  the  tests.  Specifically,  one  factor  was 
dominated  by  tests  that  involved  recognition  of  simple  patterns  (e.g.,  Canceling  Symbols  and 
Finding  e  and  ¥)  -  which  we  designated  as  PS-Pattem  Recognition  (abbreviated  PS-Pattem). 
A  second  factor  involved  scanning,  comparison,  and  look-up  processes  (e.g.,  Name- 
Comparison  Test  and  Number  Comparison  Test),  which  we  designated  as  PS-Scanning.  The 
third  factor  was  best  identified  as  making  substantial  demands  on  working  memory  (e.g., 
Digit/Symbol,  and  Coding  tests).  We  identified  this  factor  as  PS-Memory.  A  complete 
representation  of  PS  ability  also  includes  an  additional  PS  factor  identified  in  previous 
research  (called  PS-Complex,  because  the  tests  involve  both  traditional  PS  and  additional 
cognitive  components,  such  as  spatial  ability  and  estimation/interpolation,  and  heightened 
working-memory  loads;  see  Ackerman,  Kanfer,  &  Goff,  1995).  This  factor  structure  was 
replicated  in  a  later  study  (also  reported  in  Ackerman  &  Cianciolo,  2000). 

Based  on  our  research,  these  four  PS  factors  (PS-Complex,  PS-Memory,  PS-Pattem, 
and  PS-Scanning)  have  differential  associations  with  task  performance  across  skill  acquisition 
trials.  Some  (e.g.,  PS-Complex  and  PS-Memory)  are  especially  important  in  accounting  for 
individual  differences  in  performance  on  complex  skill  tasks,  while  others  (PS-Pattem  and  PS- 
Scan)  are  more  important  in  accounting  for  individual  differences  in  performance  on  simple 
skills. 


Psvchomotor  Abilities.  Although  new  developments  are  not  considered  for  the 
empirical  studies  described  below,  our  research  has  shown  that  there  psychomotor  ability 
measures  can  also  provide  incremental  predictive  validity  for  individual  differences  in  skilled 
performance  --  especially  for  tasks  that  are  routine  and  proceduralized  (see,  e.g.,  Ackerman  & 
Cianciolo,  1999;  2000).  Ability  factors  for  Tapping,  Mirror  and  Maze  Tracing,  and  for 
Serial  Reaction  Time  have  been  reasonably  well  established  as  useful  predictors  after 
extensive  task  practice. 

Experiments  Conducted 

Five  major  experiments  were  conducted  in  the  context  of  this  research  project.  Each  is 
briefly  described  below. 

Experiment  1.  The  first  study  in  this  project  was  devoted  to  validating  the  underlying 
nomothetic  constmct  space  for  perceptual  speed  abilities,  using  existing  paper  and  pencil  tests. 
In  addition,  we  added  an  assessment  of  several  measures  of  working  memory  abilities,  to 
better  integrate  the  current  constmcts  with  those  that  have  been  proposed  to  be  integral  to  the 
assessment  of  general  intelligence.  The  basic  goals  of  the  study  were  accomplished 
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(validating  the  construct  space  of  perceptual  speed  abilities).  From  a  scientific  perspective, 
this  research  also  demonstrated  that  measures  of  working  memory  are  highly  correlated  with 
perceptual  speed  abilities.  The  experiment  is  described  in:  Ackerman,  P.  L.,  Beier,  M.  E.,  & 
Boyle,  M.  O.  (2002).  Individual  differences  in  working  memory  within  a  nomological 
network  of  cognitive  and  perceptual  speed  abilities.  Journal  of  Experimental  Psychology: 
General,  131,  567-589. 


Abstract 

It  has  become  fashionable  to  equate  constructs  of  working  memory  (WM)  and  general 
intelligence  (or  g).  Few  investigations  have  provided  direct  evidence  that  working  memory 
and  g  measures  yield  similar  ordering  of  individuals.  Correlational  investigations  have 
yielded  mixed  results.  We  assess  the  construct  space  for  working  memory  and  g,  and 
demonstrate  that  WM  shares  substantial  variance  with  perceptual  speed  (PS)  constructs. 
Thirty-six  ability  tests  representing  verbal,  numerical,  spatial  and  PS  abilities,  the  Raven  test, 
and  seven  WM  tests  were  administered  to  135  adults.  A  nomological  representation  for 
working  memory  is  provided  through  a  series  of  cognitive  and  PS  ability  models.  In  addition, 
construct  overlap  between  PS  and  WM  is  further  investigated  with  attention  to  complexity, 
processing  differences,  and  practice  effects.  (Ackerman,  et  al.,  2002;  p.  567). 

F.Ypfiriment  2.  The  second  experiment  in  the  research  project  was  devoted  to  a  try-out 
of  the  first  set  of  computerized  perceptual  speed  tests  (those  involving  single  stimulus 
presentation).  Ten  new  computerized  tests  were  created  (Name  Comparison,  Divide-by-7, 
Digit-Symbol,  Number  Comparison,  Coding,  Sum-to-10,  Directional  Headings,  CA-2, 
Letter/Number  Substitution,  and  Naming  Symbols).  The  tests  were  administered  with  two 
sessions  (for  assessment  of  test-retest  reliability),  along  with  parallel  paper  and  pencil  versions 
of  the  same  tests  (for  the  assessment  of  alternate  form  reliability),  and  with  a  series  of 
cognitive  ability  measures.  A  sample  of  167  college  students  participated  in  the  experiment. 
Test-retest  reliability  estimates  were  substantial,  especially  given  that  the  computerized 
perceptual  speed  tests  are  short  and  highly  speeded.  Test-retest  reliability  estimates  ranged 
from  r  =  .72  to  .84,  with  a  mean  of  .79.  As  expected,  alternate  form  reliabilities  were  smaller 
than  test-retest  reliabilities,  but  were  quite  reasonable  in  magnitude,  indicating  that  the 
computerized  and  paper  and  pencil  measures  were  assessing  broadly  the  same  underlying 
constructs.  The  range  of  alternate  form  reliabilities  was  .45  to  .78,  with  a  mean  of  .62. 

Experiment  3.  The  third  experiment  in  the  research  project  was  designed  to  assess  the 
role  of  speed-accuracy  tradeoff  instructions  on  performance  on  the  new  computerized 
perceptual  speed  ability  measures.  A  sample  of  164  college  students  were  first  administered 
six  computerized  perceptual  speed  tests  under  standard  instructions,  followed  by 
administrations  with  either  speed  emphasis  or  accuracy  emphasis  (under  a  counterbalanced 
order  design).  Mean  performance  under  the  speed  and  accuracy  conditions  indicated  that  the 
participants  were  generally  capable  of  changing  their  performance  strategies  to  reflect  the 
change  in  speed-accuracy  emphasis,  in  terms  of  changes  to  number  correct  responses  and 
number  of  errors.  Most  significant  were  the  robust  correlations  between  individual  differences 
in  performance  across  the  two  conditions.  Speed  and  accuracy  condition  performance  levels 
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correlated  from  .51  to  .68  for  the  six  tests,  with  a  mean  correlation  of  r  =  .63.  In  addition,  by 
applying  a  standard  penalty  for  errors  to  both  conditions  resulted  in  relatively  little  differences 
in  mean  overall  performance  for  the  computerized  perceptual  speed  tests,  suggesting  that  the 
tests  provide  robust  estimates  of  ability,  even  imder  differing  emphases  for  speed  or  accuracy 
of  performance.  Results  of  this  study,  with  attention  to  gender  differences  in  performance 
under  the  different  conditions  were  presented  at  the  2003  American  Psychological  Society 
annual  meeting:  Boyle,  M.,  Beier,  M.  E.  &  Ackerman,  P.  L.  (May,  2003).  Speed-accuracy 
tradeoff,  test  performance  and  gender.  Poster  presented  at  the  annual  convention  of  the 
American  Psychological  Society.  Atlanta,  GA. 

Experiment  4.  This  study  was  designed  to  try-out  the  second  set  of  computerized 
perceptual  speed  tests  —  those  that  involve  screens  with  multiple  stimuli  (e.g.,  where  the 
examinee  searches  a  display  for  a  stimulus  that  fits  a  certain  criterion,  such  as  words  with 
both  an  ‘a’  and  a  ‘t’  in  them).  The  study  involved  160  college  student  participants,  who  were 
administered  six  new  computerized  perceptual  speed  tests  (Canceling  Symbols,  Finding  a/t, 
Summing-to-10,  Scattered  Xs,  Factors  of  7,  and  Finding  e  and  ¥).  As  with  the  previous 
experiments,  the  new  tests  were  administered  in  a  test-retest  format,  along  with  various  paper 
and  pencil  (and  computerized)  reference  tests.  Results  from  this  experiment  indicated  that 
these  new  tests  had  reliabilities  of  the  same  range  as  the  single-stimulus  item  tests. 

Experiment  5.  The  final  study  in  this  project  aimed  to  integrate  the  previous 
development  of  computerized  perceptual  speed  tests  with  extant  cognitive  ability  measures,  in 
an  effort  to  determine  the  independent  and  incremental  criterion-related  validity  of  these 
measures  of  performance  of  complex  skill  acquisition  tasks.  Fourteen  computerized 
perceptual  speed  tests  (both  single-stimulus  and  multiple-stimulus  tests)  were  administered, 
along  with  a  battery  of  twelve  cognitive  ability  measures  and  five  psychomotor  ability  tests. 
Skill  acquisition  practice  was  provided  on  two  criterion  tasks  —  28  ten-minute  trials  for  the 
Kanfer-Ackerman  Air  Traffic  Controller  Task  (K-A  ATC)  (e.g.,  see  Kanfer  &  Ackerman, 
1989)  and  15,  thirty-minute  trials  for  the  high-fidelity  Terminal  Radar  Approach  Control 
(TRACON)  simulation  task  (e.g.,  see  Ackerman,  1992).  A  sample  of  117  college  students 
participated  in  the  study.  Data  analysis  is  continuing,  but  preliminary  results  indicate  that  the 
new  computerized  perceptual  speed  ability  measures  accounted  for  12.6%  of  the  variance  in 
final  skilled  performance  in  the  K-A  ATC  task,  and  28.7%  of  the  variance  in  final  skilled 
performance  on  TRACON,  the  more  complex  criterion  task.  In  conjunction  with  the  other 
reference  ability  measures,  we  were  able  to  account  for  a  total  of  40.0%  of  the  variance  in 
final  K-A  ATC  task  performance,  and  38.9%  of  the  variance  in  final  TRACON  performance. 
These  results  clearly  indicate  that  efficacy  of  the  new  perceptual  speed  measures  for 
enhancing  the  prediction  of  individual  differences  in  complex  task  performance. 

Conclusions/Recommendations 

Given  the  large  investment  made  in  both  selection  and  training  of  U.S.  Air  Force 
persoimel,  improvements  to  the  accuracy  of  prediction  of  individual  differences  in 
performance  (especially  those  that  are  substantially  cost-effective)  seem  to  present  an 
opportunity  for  further  research  and  implementation  in  the  operational  environment.  In  this 
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research  project,  we  have  developed,  refined,  and  demonstrated  both  the  reliability  and 
validity  (for  predicting  individual  differences  in  complex  task  performance)  of  a  set  of 
perceptual  speed  ability  tests.  These  tests  are  fully  computerized  and  can  be  administered  in 
an  off-the-shelf  PC  computer  environment.  The  individual  tests  are  short  (each  test  requires 
only  a  few  minutes  to  aininister),  and  are  robust  to  differential  strategies  of  speed  or 
accuracy  emphasis.  With  the  existing  software  platform,  the  tests  can  be  modified  with  little 
effort  (which  with  further  development,  could  be  modified  in  real-time),  to  avoid  any 
problems  associated  with  repeated  assessments  or  test  practice.  The  measures  demonstrate 
substantial  incremental  validity,  over  and  above  the  kind  of  extant  tests  of  cognitive  and 
intellectual  abilities  that  underlie  the  standard  selection  measures  used  by  the  U.S.  Air  Force. 
Further  exploration  of  these  tests,  or  tests  similar  to  those  developed  in  this  project,  is  highly 
recommended  for  ultimate  adoption  in  the  operational  selection  environment. 
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